Systems and methods for facilitating enhancements to search engine results

ABSTRACT

In accordance with some embodiments, processes and interfaces for enhancing or modifying the output of search results are provided. For example, an interface may allow a user to indicate a particular search result returned for a search as being undesirable. In one embodiment, such an indication may cause a removal (e.g., by a search engine or by a third party software application with access to a listing of search results returned by the search engine) of the search result from the listing of the search results as well as review of the remainder of the search results to determine which additional search results should also be removed (e.g., additional search results may be determined, on behalf of the user, to be undesirable for the same reason as the search result(s) indicated by the user).

The present Application claims the benefit of U.S. ProvisionalApplication Ser. No. 61/481,236, filed on May 1, 2011 in the name ofAlan Reznik and entitled Methods and Systems for Search EngineEnhancements. The entirety of this Application is incorporated byreference herein for all purposes.

The present Application is related to U.S. Non-Provisional applicationSer. No. 13/326,593, filed herewith in the name of Alan M. Reznik andentitled SYSTEMS AND METHODS FOR FACILITATING ENHANCEMENTS TO ELECTRONICGROUP SEARCHES. The entirety of this application is incorporated byreference herein for all purposes.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic diagram of one embodiment of a system operable tofacilitate some methods described herein.

FIG. 2 is Venn diagram illustrative of some advantages which may berealized via some methods described herein.

FIG. 3A is a diagram of one interface via which a user may be providedwith search results utilizing a first search engine after inputting aparticular search term, in accordance with the prior art.

FIG. 3B is a diagram of one interface via which a user may be providedwith search results utilizing a second search engine after inputting thesame search term utilized in FIG. 3A, in accordance with the prior art.

FIG. 3C is a diagram of one interface via which a user may be providedwith search results utilizing a third search engine after inputting thesame search term utilized in FIG. 3A and FIG. 3B, in accordance with theprior art.

FIG. 3D is a diagram of one interface via which a user may be providedsearch results utilizing the first search engine of FIG. 3A but afterinputting a different search term, in accordance with the prior art.

FIG. 3E is a diagram of one interface via which a user may be providedwith search results utilizing the search engine of FIGS. 3A and 3D, thesearch utilizing a method available in the prior art to refine thesearch for the term utilized in FIG. 3D.

FIG. 4 is a flow diagram illustrating a process which may be performedin accordance with some embodiments described herein.

FIG. 5 is a flow diagram illustrating a process which may be performedin accordance with some embodiments described herein.

FIG. 6 is a flow diagram illustrating a process which may be performedin accordance with some embodiments described herein.

FIG. 7 is a diagram of one example interface, consistent with someembodiments described herein, via which a user to whom a set of searchresults is output may indicate undesirable search results to be removedfrom the set of search results.

FIG. 8 is a diagram of another example interface, consistent with someembodiments described herein, via which a user to whom a set of searchresults is output may indicate undesirable search results to be removedfrom the set of search results.

FIG. 9 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user to whom a set ofsearch results is output may indicate undesirable search results to beremoved from the set of search results.

FIG. 10 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user to whom a set ofsearch results is output may indicate undesirable search results to beremoved from the set of search results.

FIG. 11 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user to whom a set ofsearch results is output may indicate undesirable search results to beremoved from the set of search results.

FIG. 12 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user to whom a set ofsearch results is output may indicate undesirable search results to beremoved from the set of search results.

FIG. 13 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user is presented withuser directed feedback options to search results, as well as additionalinformation.

FIG. 14 is a diagram of yet another example interface, consistent withsome embodiments described herein, via which a user is presented withuser directed feedback options to search results, as well as additionalinformation.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Certain aspects, advantages, and novel features of the invention aredescribed herein. It is to be understood that not necessarily all suchadvantages may be achieved in accordance with any particular embodimentof the invention. Thus, for example, those skilled in the art willrecognize that the invention may be embodied or carried out in a mannerthat achieves one advantage or group of advantages as taught hereinwithout necessarily achieving other advantages as may be taught orsuggested herein.

Although several embodiments, examples and illustrations are disclosedbelow, it will be understood by those of ordinary skill in the art thatthe invention described herein extends beyond the specifically disclosedembodiments, examples and illustrations and includes other uses of theinvention and obvious modifications and equivalents thereof. Embodimentsof the invention are described with reference to the accompanyingfigures, wherein like numerals refer to like elements throughout. Theterminology used in the description presented herein is not intended tobe interpreted in any limited or restrictive manner simply because it isbeing used in conjunction with a detailed description of certainspecific embodiments of the invention. In addition, embodiments of theinvention can comprise several novel features and it is possible that nosingle feature is solely responsible for its desirable attributes or isessential to practicing the inventions herein described.

Embodiments described herein are directed to methods, systems andinterfaces for providing one or more enhancements to a search engine,search algorithm or other search tool, such that results output by thesearch tool may be modified, refined and/or filtered based on an inputfrom a user, such as an input indicating which of the results areunwanted or undesired by the user. Such methods, systems and interfacesfurther provide for, in accordance with at least one embodiment,allowing a user to more efficiently identify wanted but unknowninformation which may be buried in a deluge of search results. Forexample, a search engine may perform a search of available informationbased on one or more search terms input by a user and present an initiallist of search results to a user via an interface. This interface mayallow the user to “check off” or otherwise indicate which one or more ofthe results which have insufficient current value to the user. Thesearch engine may then (i) rerun the search based on that “negativefeedback” to identify a modified list of results, which presumable willnot include the one or more results indicated by the user as unwanted orundesired (and/or other similar results); and/or (ii) modify the initiallist of search results by removing not only the result(s) indicated bythe user as unwanted or undesired but also other similar results. Ineither case, the goal is to shorten the list of results the user needsto consider or review. In scenario (ii), the other similar results maybe identified based on a characteristic they share with the particularresult the identified by the user as being unwanted or undesirable.

In another example, a user requesting a search (e.g., by inputting oneor more search terms) may further indicate (e.g., prior to launching orinitiating the search) one or more rules or criteria defining unwantedor undesired search results. For example, the user may indicate thathe/she considers the top 5 most popular results to be undesirable. Suchan indication of one or more rules or criteria may be utilized by asearch tool, in one embodiment, as a filter for any search results foundbased on one or more search terms input by the user, the filter forfiltering out which search results are not to be output to the user. Forexample, in one such an embodiment, a search engine (or another deviceoperable to perform some of the methods described herein) may remove anysearch results that satisfy the rule or criteria identified by the user.In some embodiments, the removal may be done prior to outputting to theuser the search results, such that the user is not shown the removedsearch results as part of listing of search results (e.g., unless theuser selects an option to see such removed search results, as may beavailable in some embodiments).

It should be noted that an unwanted or undesired search result need notbe a result that is not relevant or appropriate based on the searchterm(s) provided by the user for the initial search. In fact, it may bethat a result that the user indicates as undesirable or unwanted may bea result ranked as highly relevant and/or popular by the search enginebeing used for the search. An unwanted or undesired search result issimply a result that the user prefers not to have included in the listof search results for a particular search, for any reason. Applicant hasrecognized that the results of searches carried out by use of searchengines, tools and algorithms available today comprise such a large setof results that in many circumstances it is unlikely that a user whorequested the search will have the time, desire or ability to view eachof the results in the set. Applicant has further recognized that thisinability or unwillingness to view all of the search results may resultin the user missing out on learning some important information that maybe “buried” in the large amount of search results. Embodiments describedherein provide for a strategy and interface to allow a user of a searchtool or resource (e.g., the common search engine user) better access andcontrol over search queries and the results. Applicant has recognizedthat most users have an intuitive ability to better discern what is notwanted more quickly and decisively than what is wanted. In recognitionof this human ability, at least some embodiments described hereincomprise a process which provides one or more mechanisms for removingunwanted results efficiently as a method of obtaining a better result.Further, in at least some embodiments described herein a modified set ofsearch results is obtained for a user who indicates which results of aninitial set of search results are unwanted by not only removing thespecifically selected results but further modifying the search to removeother similar results that are determined by the system to also beunwanted or likely to be unwanted based on the user input. Suchembodiments may be contrasted with prior art search methodologies whichrely on positive feedback (“more like this”) from a user or which onlyallow a user to modify a search by removing a single search result at atime or by manually modifying the specific search terms used andrerunning the search. This is further contrasted from prior art thatuses the aggregate of what other users found desirable when they havesearched for a similar term or query to determine the information thecurrent user values. This ubiquitous process further limits a given userfrom finding new or novel information in a given search when new andnovel information or associations are desired by the user.

It should be noted that in some embodiments an identification of aparticular search result may be received from a user to whom an initiallist of search results is output in response to search terms provided,and a determination may be made by a processor (e.g., a processorrunning a search algorithm or search engine or a processor of anotherdevice that is operable to communicate with a search engine) ofadditional search results to be removed based on an identification ofthe particular search result. Such a determination of additional searchresults to be removed may be made, in accordance with some embodiments,based on a characteristic of the particular search result identified bythe user. The characteristic may be derived or identified by theprocessor without input from the user as to the characteristic or,alternatively, the characteristic may be identified (e.g., selected froma menu of possible characteristics, typed in or otherwise specified) bythe user.

In accordance with one embodiment, a method provides for (i) determininga first plurality of search results of a search performed by a searchtool; (ii) receiving an input of a user associated with the firstplurality of search results, the input indicating at least one result ofthe first plurality of search results to be removed from the firstplurality of search results, thereby receiving an indication of at leastone first undesired search result of the first plurality of searchresults; (iii) determining a remainder of the first plurality of searchresults to be the first plurality of search results less the firstundesired search result; (iv) identifying a characteristic of the atleast one first undesired search result; (v) analyzing the remainder ofthe first plurality of search results to determine whether any searchresults of the first plurality of search results correspond to thecharacteristic; (vi) determining at least one search result of theremainder of the first plurality of search results that corresponds tothe characteristic to be a second undesired search result; (vii)removing the first undesired search result and the second undesiredsearch result from the first plurality of search results, therebydetermining a second plurality of search results; and (viii) causing thesecond plurality of search results to be output.

In accordance with another embodiment, a method provides for (i)determining a first plurality of search results of a search performed bya search tool; (ii) receiving an input of a user associated with thefirst plurality of search results, the input indicating a preference fora maximum popularity ranking of a search result, in accordance with aranking scheme in which a higher popularity ranking indicates a morepopular search result than does a lower popularity ranking; (iii)analyzing the first plurality of search results to identify any searchresults corresponding to a popularity ranking higher than the maximumpopularity ranking; (iv) removing from the first plurality of searchresults, any search results that correspond to a popularity rankinghigher than the maximum popularity ranking, thereby determining a secondplurality of search results; and (v) causing the second plurality ofsearch results to be output.

In accordance with another embodiment, a method for facilitating a groupresearch project to be managed in a manner that encourages more diversesearch results than may otherwise be obtained using conventional searchmethods which allow all members of the group to simply search theInternet or another data source without restriction (as is explained inmore detail below). Such a method provides for (i) registering aplurality of users as a search group for a search project; (ii)designating one user of the plurality of users as a manager for thesearch project, wherein the manager for the search project is authorizedto set search rules for the search project which govern an output ofsearch results to the remainder of users of the search group; (iii)receiving, by a processor of a computing device operable to modify a setof search results and from one of the plurality of users, an indicationof at least one search result to be removed from a set of search resultsoutput to another user of the search group, thereby receiving anindication of at least one disallowed search result; (iv) determining,by the processor, a preliminary set of search results based on thesearch term, the preliminary set of search results comprises results ofa search of information available on the internet; (v) determining, bythe processor, whether the disallowed search result is included in thepreliminary search results; (vi) if the disallowed search result isincluded in the preliminary search results, removing, by the processor,the disallowed search result from the preliminary search results,thereby determining a modified set of search results; and (vii) causingthe modified set of search results to be output to at least the userfrom whom the search term was received.

In accordance with one embodiment for facilitating a group researchproject, a method provides for (i) registering a plurality of users as asearch group for a search project; (ii) tracking sets of search resultsoutput to users of the search group for the search project; and (iii)outputting, to a user of the search group an indication of a pluralityof search results included in any of the set of search results alongwith, for each such search result, at least one of (i) a number of timesthe search result has been included in a set of search results output tothe users of the search group; (ii) a duration of time a single user ofthe search group has spent reviewing a content of the search result;(iii) a total duration of time all users of the search group who had thesearch result output to them in a set of search results reviewing thecontent of the search result; (iv) a number of time the search resultshas been selected by a user of the search group; and (v) present amathematical assessment of the search result output or page use (e.g.,an average or median time spent by one or more users on a given searchresult, a number of repeat views of a given search result by one or moreusers or an average user).

It should be noted that any of the methods and processes describedherein may be performed, in some embodiments, by one or more processorsof one or more a computing devices operable to modify search results.For example, in some embodiments, one or more servers of a search engineprovider may perform one or more processes described herein. In anotherembodiment, a server of an intermediary service which serves tofacilitate searches requested by a user and performed by a search engineprovider may perform one or more of the processes described herein. Inyet another embodiment, a software application may be downloaded to auser's computing device (e.g., a mobile device) may be operable toperform one or more processes described herein. For example, a user maydownload an “app” for allowing the user to filter out or remove searchresults in accordance with one or more of the embodiments describedherein.

EXAMPLE SYSTEMS AND APPARATUS

Referring now to FIG. 1, illustrated therein is schematic diagram of anexample system 100 which may be utilized to provide some of theprocesses described herein. The system 100 comprises a plurality of userdevices 102, 104 and 106 and a plurality of search tool servers 112, 114and 116. In accordance with some embodiments, system 100 furtherincludes a third party server 118, which may comprise a server whichfacilitates search requests from and/or output of search result to anyor all of the user devices 102, 104 and 106. Any of the devices 102,104, 106, 112, 114, 116 and 118 may in accordance with some embodimentsbe operable to communicate with at least one other device of system 100via a network 110. The network 110 may comprise, for example, theInternet, a wide area network, another network or a combination of suchnetworks. It should be understood that although not shown in FIG. 1,other networks and devices may be in communication with any of thedevices of system 100. For example, a user device 102 may comprise amobile device which may be in communication with a search tool server112 via a mobile network (not shown) such as a pager or cellulartelephone network that accommodates wireless communication with mobiledevices as is generally known to those skilled in the art.

A search tool, as the term is used herein, may comprise a mechanismcomprising software, hardware and/or firmware operable to electronicallysearch through a large quantity of available data (whether private orpublicly available data) to identify or determine subset(s) of the datathat may be relevant to one or more search terms or criteria provided bya user of the search tool. One type of search tool contemplated hereinis a search engine, which may comprise an algorithm designed to searchthrough information available on the Internet and/or a private network.Many search engines are designed to index a large number of web pages orother formats of information and provide an interface via which a usermay search the indexed information by, for example, inputting one ormore terms and/or rules defining a particular search. A search enginemay, for example, be operable to utilize a “crawler” mechanism which“crawls” the Internet to locate documents or other formats ofinformation and store data about such documents or other informationsuch that these may be located and/or searched at a later time (e.g.,the search engine may store the URL, associated hyperlinks, meta-data,etc.). Many search engines utilize extraction and/or indexing mechanismsfor extracting and/or indexing information about the documents, webpages or other formats of information accessible to them. Such indexingmay be based on the contents of the documents, web pages or otherinformation and the indexed form of the information may be stored in adatabase of the search engine. The search engine may comprise a searchtool component which allows a user of the search engine to searchthrough the indexed contents of the database. A search engine may alsobe operable to output the results of any search of the database (thesearch having been conducted in accordance with the search terms and/orrules provided by the user), may times in accordance with a particularranking scheme programmed into the search engine. For example, manysearch engines rank the most popular search results associated with aparticular search term higher than less popular search results.

As described above, in some embodiments a search tool or search enginemay be operable to search a database of available information. Thedatabase may, in some embodiments, be proprietary to the search tool. Inother embodiments, the database may be a shared database. In eitherscenario, the database may be stored by one or more search tool servers112, 114 and 116 and/or a third party server 118 (e.g., in a memory ofthe appropriate server), on the Web or in Cloud Storage (e.g., in amemory of a server distinct from the search tool server accessing thedatabase) or in some alternate data storage location. Further, while insome embodiments such a database of information may have been generatedby the search tool or third party server accessing the database, inother embodiments the database may have been created by another entityor server. Finally, although reference has been made to a search tool orsearch engine searching a database in order to determine search results,it should be understood that such a data storage scheme is not necessaryto carry out the embodiments described herein and other data storageschemes may be desirable or appropriate. For example, in someembodiments a search tool or search engine may be operable to searchthrough data or information stored in spreadsheet, a particular file ordocument and/or a hierarchical or other file storage scheme.

Any and all of the search tool servers 112, 114 and 116 (as well asthird party server 118) may comprise one or more computing devices,working in parallel or series if more than one, operable to facilitatethe enhanced search result modification functions described herein. Anyand all of the user devices 102, 104 and 106 may comprise, for example,a computing device operable to receive input from a user regarding asearch to be conducted (e.g., one or more search terms to be used inconducting the search and/or one or more characteristics, rules orcriteria to be used in modifying the results of the search as they areto be output to the user).

In some embodiments, a user device such as a user device 102, 104 or 106may comprise a portable computing device such as a smartphone (e.g., theIPHONE or IPAD manufactured by APPLE, the BLACKBERRY manufactured byRESEARCH IN MOTION, the PRE manufactured by PALM or the DROIDmanufactured by MOTOROLA), a Personal Digital Assistant (PDA), cellulartelephone, laptop or other portable computing device. In otherembodiments, a user device may comprise a desktop, kiosk, ATM or othernon-portable computing devices. In some embodiments, a user device maybe a dedicated device dedicated to outputting search results andreceiving user feedback regarding such search results. In someembodiments, a user device may comprise a simplified computing devicewith limited processing power, such as a digital picture frame or aCHUMBY device. In some embodiments, a user device may be integrated intoanother system, packaging, structure or device, such as a vehicle,wearable apparel, entertainment system and/or be operable to dock orconnect with a wireless enabling accessory system (e.g., a Wi-Fi dockingsystem). In some embodiments, a user device may be operable tosynchronize with a local or remote computing system to receive, downloador upload content, download software applications and to receive and/ortransmit other data.

It should be understood that any or all of the devices of system may insome embodiments comprise one or more of (i) an input device; (ii) anoutput device; (iii) an input/output device; or (iv) a combinationthereof.

An input device, as the term is used herein, may be any device, elementor component (or combination thereof) that is capable of receiving aninput (e.g., from a user or another device). An input device maycommunicate with or be part of another device. Some examples of inputdevices include: a bar-code scanner, a magnetic stripe reader, acomputer keyboard or keypad, a button (e.g., mechanical,electromechanical or “soft”, as in a portion of a touch-screen), ahandle, a keypad, a touch-screen, a microphone, an infrared sensor, avoice recognition module, a coin or bill acceptor, a sonic ranger, acomputer port, a video camera, a motion detector, a digital camera, anetwork card, a universal serial bus (USB) port, a GPS receiver, a radiofrequency identification (RFID) receiver, an RF receiver, a thermometer,a pressure sensor, an infrared port, and a weight scale.

An output device may comprise any device, component or element (or acombination thereof) operable to output information from any of thedevices described herein. Examples of an output device include, but arenot limited to, a display (e.g., in the form of a touch screen), anaudio speaker, an infra-red transmitter, a radio transmitter, anelectric motor, a dispenser, an infra-red port, a Braille computermonitor, and a coin or bill dispenser.

An input/output device may comprise components capable of facilitatingboth input and output functions. In one example, a touch-sensitivedisplay screen comprises an input/output device (e.g., the deviceoutputs graphics and receives selections from an authorized person).

It should be understood that each of the devices of system 100 maycommunicate with one another directly or indirectly, via a wired orwireless medium such as the Internet, LAN, WAN or Ethernet, Token Ring,or via any appropriate communications means or combination ofcommunications means. For example, in one embodiment communication amongany and all of the devices of system 100 may occur over the Internetthrough a Web site maintained by computer on a remote server or over anon-line data network including commercial on-line service providers,bulletin board systems and the like. In yet other embodiments,communication among any of the devices of system 100 may occur over RF,cable TV, satellite links and the like.

The system 100 may be operable to facilitate communication among thedevices comprising the system using known communication protocols.Possible communication protocols that may be part of the system 100include, but are not limited to: Ethernet (or IEEE 802.3), ATP,BLUETOOTH, HTTP, HTTPS and Transmission Control Protocol/InternetProtocol (TCP/IP). Communication may be encrypted to ensure privacy andprevent fraud in any of a variety of ways well known in the art, some ofwhich are described herein.

It should be understood that although only three user devices 102, 104and 106 and three search tool servers 112, 114 and 116 are illustrated,any number of such devices may be used and, in many embodiments, a largeor smaller number of each such device would be part of system 100. Itshould further be understood that although a system 100 comprisingmultiple devices is illustrated, such a system is not necessarily (andprobably is not) operated or maintained by a single entity. For example,each of the search tool servers 112, 114 and 116 may be operated by adistinct entity (e.g., search tool server 112 may be operated by GOOGLE,search tool server 114 may be operated by BING and search tool server116 may be operated by YAHOO!) and each of the user devices 102, 104 and016 may be operated by a different user. In some embodiments, the searchresult modification functions described herein may be made availableonly from a particular search tool provider (e.g., GOOGLE). In such anembodiment, only one search tool server may be appropriate or each ofthe search tool servers 112, 114 and 116 may be operated by or on behalfof the same entity.

Described herein are various processes, features, interfaces andmechanisms for allowing a user to refine or modify a set of searchresults by providing feedback such as indication(s) that one or moreparticular search result(s) are considered undesirable and removing froma listing of search results the indicated search result(s) along withadditional search result(s) that are determined to be associated with,corresponding to, relevant or likely to be undesirable for the samereason(s) as the indicated search result(s). In some embodiments, asearch tool server such as a search tool server 112, 114 and/or 116 maybe programmed and modified to facilitate such processes, features,interfaces and mechanisms directly (e.g., the processes described hereinmay be integrated into a search engine). For example, a web browserinterface of a search tool as it is output to a user may be modified toallow the user to provide indications of which search results areundesirable (and/or to “undo” such indications) and the associatedsearch tool server may be operable to not only perform a searchrequested by the user but also refine or modify the listing of searchresults based on the user's feedback or instructions. In otherembodiments, however, such processes, features, interfaces andmechanisms may be embodied in a software application that resides on athird party server 118 which serves as an intermediary service whichfacilitate the output (and modification) of search results determined bya search tool to a user. In yet other embodiments, such processes,features, interfaces and mechanisms may be embodiment as a softwareapplication that resides in a memory of a user device (e.g., a user maydownload or install the software application onto his device), whichsoftware application may be programmed to either automatically launchupon detecting certain triggering events (e.g., when a user opens a webbrowser, when a user navigates the browser to a search tool URL, etc.)or when it is affirmatively launched or opened by the user. For example,in some embodiments the features described herein may be embodied as an“app” that is downloadable onto a user device such as an ANDROID, IPHONEor IPAD. In the group research embodiments described herein, such an appmay allow a user to link his search results to those of a group (e.g., aclassroom, library, business or other setting). In some embodiments asearch tool server, third party server or user device may store in itsmemory a program for facilitating at least some of the processes andembodiments described herein.

Description of Context for Some Embodiments

Referring now to FIG. 2, illustrated therein is a Venn diagramillustrating some advantages or benefits that may be realized inutilizing the search result modification functions described herein.Applicant has recognized that the hallmark of the best search engines isthe ability to narrow the massive amount of information remotely relatedto the target query into a tractable list. The list needs to be both anaccurate representation of the search request and short enough to beevaluated by the user who requested the search within a practical timeframe. Therefore, speed and accuracy are two of the many attributesexpected from current search technology. Speed, in particular, is judgedas quick enough to be timely for the user of the search engine. Timely,in turn, is measured in less than a few seconds. Accuracy is measured bythe closeness of the result to the wanted information. Still, speed andaccuracy are not enough. Inclusivity is also important. To be inclusivemeans the search results contain all information the user may alreadyknow about as well as other information that may be of value to the userthat the user is yet to discover. The discovery of information that isof value and not known to exist prior to the search could be termed“serendipity.” It could be argued that the web and other data bases havecreated the most value by giving the general user access to both theinformation that is known to be available and that information that wasnot known to be available. In other words, there is real value inserendipity.

Search speed, accuracy, inclusiveness and serendipity are of importanceto most users and this, for the most part, is the ultimate goal ofsearch services such as GOOGLE, BING, YAHOO! and BADU. These and manyother search engines for internet acquirable information, internet databases and non-internet database searches are after the same goal, namelyall the factors mentioned and hence a satisfying user experience.Furthermore, one could argue that without all of these factors speed,accuracy, inclusiveness and serendipity the enormous success of thecurrent engines would not have occurred. Moreover, the astronomicalwealth of information on the web would not be accessible to the averageuser without these search engines. Still, the wealth of information onthe web is ever expanding and the actions associated with inclusivenessand serendipity can often be in direct conflict with speed and accuracy.As a direct result, to be truly inclusive and allow for serendipity manysearch engines report back tens or hundreds of thousands (sometimes evenmillions) of results. The end result is an intractable list and a netloss of the available serendipity because few users can wander past thefirst ten to twenty results.

The value of serendipity cannot be truly estimated. Knowing how closelyserendipity and discovery are linked, it is hard to quantify what is newinformation that was actively searched for and what is new informationthat was unexpected. It is equally difficult to imagine a world withoutsome level of serendipity. Still, in an effort to improve searchesaccuracy is increased and serendipity is often sacrificed. In the mostsophisticated searches accuracy is judged by laser sharpness leavinglittle room for serendipity. Many other engines are judged by supplyingthe most popular results again reducing serendipity.

In order to better understand the relationships, illustrated in FIG. 2is a Venn diagram 200 showing the sets of information that represent ageneric version of all data available and the common subsets of possiblesearch results. The figure shows the relationship between the pool ofall information available/searchable via a particular search tool (202),the set of search results for a particular search request (204) as wellas a representation of the subset of search results which comprisewanted results (210) as circles in the diagram 200. The diagram 200 alsoincludes a subset of unwanted information 206 contained within thesearch results 204 and the smaller sets of known wanted information(210) and unknown wanted information or serendipity (208). In this waywe can see the union of the sets of wanted known information (206) andserendipity (208) contains the search results most likely to bedesirable to many users. The intersection of the two sets (212) is theoverlap of wanted known information (210) and wanted yet unknowninformation (208) is of special interest in expanding one's knowledge.Also illustrated via the diagram 200 is how the prior art search toolsleave some of the wanted information out. These are shown in the areasinside the sets of wanted information (210) and serendipity (208) thatlie outside the set of search results (204) and are indicated as areas(214) and (216) respectively. From these sets in the diagram 200 one cansee that there is a need for a search result modification mechanism(such as ones described herein) which can effectively remove (or allow auser associated with the search results to remove) the unwanted subsetof the search results (206) that is not in the set of known wantedsearch results (210) and the set of serendipity (unknown yet wantedresults) (208). There is further a need for a search result modificationmechanism (such as ones described herein) which allows a user toefficiently identify the data in each of the sets (208) and (210) thatis not in the original search result (204), the information which wouldbe includes in the areas (214) and (216).

To solve the conflict between providing all accessible information to beinclusive and still be accurate many search engines today are programmedto rank the most popular results higher than the most novel results in agiven list of search results out put to a user (i.e., the most popularsearch results appear first in the list of search results). This causesa further loss of serendipity and some loss of accuracy as defined by anindividual user's search requirements. This, in turn, yields anunsatisfying experience for the user in their search of new information.This frustration is an experience we can all relate to, thousands ofresults but nothing of true interest. The user is then forced to wanderaround in the set of unwanted search results (the results represented byarea 206 of diagram 200). This leads many users conducting a search tothink hard about what search result they are looking for and anotherattempt at getting it “right.” Many users continue on frustrated andless and less willing to look further down a list of search results asthe time required to find what they want increases and the value of thesearch activity decreases. Lost in the large set of unwanted results,any chance at a great answer to the user's question diminishes alongwith any real opportunity for a serendipitous finding of value.

As current engines look for a solution for this issue, the manythousands of results are ranked in several ways. Most commonly by whatother searchers may think combined with algorithm ranking, objects thatare highly linked and even your own prior searches or search behaviorsin prior searches. This practical application of technology has createda bias to what others are looking for and this, as we all know, isself-reinforcing since few searchers move past the first ten (10)results when presented with literally thousands to review. So, theproblem remains. How does one get a search engine to deliver theinformation that is most useful?

Applicants has recognized, however, that the human mind can more quicklyreject a poor search result than a good one. In other words, the userwho has input search terms for a search and been provided with searchresults has a natural ability for a sharper distinction about what theydo not want than what they do want. Applicant has further recognizedthat it is the elements of what is not wanted in a search that oftenkeeps the search from being a successful search. Seeing the volume ofunwanted information in the search results is an unsatisfyingexperience. Applicant has thus recognized that if a user were givenproper tools or a set of tools for easily removing the noise or unwantedinformation, finding the information the user wants would be simplifiedand more satisfying.

Turning now to FIGS. 3A through 3C, illustrated therein are respectiveexample user interfaces illustrating prior art search results for theindicated search term “181 E 85th St New York, N.Y. 10028”. FIG. 3Acomprises a user interface depicting search results output to a userusing the GOOGLE search engine after the user input the search term “181E 85th St New York, N.Y. 10028.” As indicated on FIG. 3A, over one (1)million results were returned to the user. FIG. 3B comprises a userinterface depicting search results output to a user using the YAHOO!search engine after the user inputs the same search term “181 E 85th StNew York, N.Y. 10028.” As indicated on FIG. 3B, about 527,000 resultswere returned to the user. FIG. 3C comprises a user interface depictingsearch results output to a user using the BING search engine after theuser inputs the same search term “181 E 85th St New York, N.Y. 10028.”As indicated on FIG. 3C, about 474,000 results were returned to theuser. It should be noted that, due to page space constraints, only thefirst few results output to the user on the first web page of searchresults are shown in each of the figures. It should further be notedthat the search results illustrated in each of the figures are based onan actual search using the search term “181 E 85th St New York, N.Y.10028” conducted on each respective search engine in the year 2011.

FIG. 3D comprises a user interface depicting search results output to auser using the GOOGLE search engine after the user inputs the searchterm “President Clinton first year in the white house.” As indicated onFIG. 3D, over 1.4 million search results were returned to the user. FIG.3E comprises a user interface depicting search results output to a userusing the GOOGLE search engine after the user utilized GOOGLE's advancedoperators to try to remove known unwanted information after viewing thefirst search results output after an initial search based on the searchterm “President Clinton first year in the white house.” In consideringthe massive quantity of results returned by each of the search enginesfor a fairly specific search term, as illustrated in FIGS. 3A through3D, one can appreciate that unwanted information can easily overwhelmthe results and create significantly distracting “noise” that may keep auser from efficiently finding the wanted results. One can alsoappreciate that most users, in viewing the results on a particular pagewould likely be able to easily determine which results are not wanted orundesired.

FIG. 3E illustrates the result of attempting to remove unwantedinformation returned in an initial search using an available searchengine mechanism. As shown in FIG. 3E, rather than decreasing the numberof search results such that the set of search results is more manageablefor the user, the refined search yielded many more search results (overtwo (2) million this time). Thus, using the advanced search feature on asearch engine such as GOOGLE by taking advantage of its algorithm ormethod to fine tune the search returns somewhat paradoxically almosttwice as many search results. This is of interest since the effort toreduce or focus results using the current state of the art actuallyyields more results not less results. This is the opposite of what onewould expect. This paradoxical result of more results even when tryingto remove the unwanted results further demonstrates the need in thestate of the art for an alternative strategy. This paradoxical resultcould only be explained by the search engine's net bias to being allinclusive and its general inability to use negative factors in narrowingresults by nature of their inherent design.

Therefore, there is a need in the state of the art for an improved userdirected search result strategy or interface that is user friendly,intuitive, search engine or algorithm independent, and non-intrusive andwhich provides the benefits of vast data search strategies while beingfast, accurate, inclusive and further provides an opportunity forserendipity while allowing the user to self direct the elimination ofextraneous information in both a negative context (undesirableinformation) and even a positive context (more desirable information). Anew search enhancement or tool and interface of this type may have oneor more of the following benefits:

-   -   (i) present to the user clear easy options to modify search        results in order to allow the user to quickly and efficiently        remove the unwanted results as well as all similar unwanted        material;    -   (ii) improve the user's view of the wanted data by removing the        “noise” created by the unwanted data;    -   (iii) indirectly create access to the unknown but useful        information, in other words increasing the chance for        serendipity, by lessening the “noise” in the result;    -   (iv) improve the relevance of searches and therefore increase        the so called “stickiness” (the tendency to stay as a user of        the engine or portal and revisit it for future searches) of a        given search site. This would increase the value of the site for        the various stakeholders including but not limited to the user,        the search provider, the content providers including the        potential revenue sources such as paid search, advertises and        other content providers;    -   (v) allow the searcher to indirectly create and use the power of        a more sophisticated Boolean type search in a user friendly        intuitive way;    -   (vi) remove the current limits placed on these modification by        providing a better operational application of the user supplied        information;    -   (vii) allow the search engine to heuristically acquire        information about information that detracts from a given search        directly from the users and improve the accuracy and value of        the search outcome to the individual searcher;    -   (viii) allow for the creation of URL based or Logged in based        tracking of search detractors for a given computer or individual        searcher (the logged in version would allow the prior history to        “follow” the logged in user from device to device); and/or    -   (ix) enhance the value of the search engine by “knowing” the        important factors to a searcher to create value in a logged in        (user identified) environment like those created when a user        maintains an account or user ID with a search engine or portal.

Accordingly, described herein are various embodiments for strategies,processes, systems and interfaces which are operable to allow a searchtool user better access and control over search queries and the results.The interfaces, processes and systems described herein, and theassociated functionalities are search tool independent. Described hereinare various examples for orders of operations search, applications ofsearch modifying options over the current search results, optionsselections, application of modifications, the ability to “undo” or“redo” an applied option and the ability to repeat these sets in aniterative fashion as well as alternative ordering of the same functionaloperations. In accordance with some embodiments, a user may input aquery through any acceptable interface, the search tool or search enginemay present the search results to the user and provide a direct feedbackoption to the user, via which the user may indicated unwanted orundesirable search results. A goal of at least some embodimentsdescribed herein is to provide systems, methods and interfaces operableto aid a user of a search tool to remove unwanted results or “noise”efficiently. The search result modification options described hereincomprise an effective means to reduce the noise and bring the morepertinent results closer to the top of the list while reducing the listsufficiently enough to allow for serendipity (as described withreference to FIG. 2 herein). The user thus should be able to enhance thepossibility of viewing more pertinent data and identifying informationthat they were unaware of prior to the application of the usermodifications. The dynamically applied negative as well as potentialpositive feedback created by the user feedback interfaces describedherein may, in some embodiments, (i) be stored to help future usersearches, and/or (ii) be tracked by a computer's location (URL) or auser login to better assist in future searches (of the same user and/ordifferent users). In accordance with some embodiments, one or more ofthe user directed modifications can be added to the search enginedatabase as it improves its own methods of searches in order to reduce“noise” or unwanted data and improve serendipity with the goal of abetter user experience.

Example Processes

Referring now to FIG. 4, illustrated therein is a flowchart of a process400 consistent with some embodiments described herein. It should benoted that process 400 is exemplary only and should not be construed ina limiting fashion. For example, additional and/or substitute steps tothose illustrated may be practiced within the scope of the presentinvention and in one or more embodiments one or more steps may beomitted or modified. In one embodiment, the process 400 is performed bya search tool server 112, 114 or 116. Alternatively, process 400 may beperformed by a third party server 118 and/or a software applicationresiding on or otherwise utilized by (e.g., using cloud storagetechnology) a user device of a user requesting the search.

In step 402, an initial set of search results for a given search isdetermined. For example, if step 402 is being performed by a search toolserver 112, 114 or 116, step 402 may comprise performing a search ofavailable data based on one or more search terms and/or rules defining asearch (e.g., as input by a user) and identifying the highest rankedsearch results for the search based on one or more algorithms. If, onthe other hand, the step 402 is performed by a third party server 118 orby a software application of a user device, step 402 may compriseidentifying the search results returned for a particular search by asearch tool (e.g., as output via a search engine web browser interface).In either case, the step 402 may in some embodiments be preceded by auser inputting one or more search terms and/or rules defining a searchinto a search tool interface, which user input may initiate a search andresult in an initial set of search results for the search. In oneembodiment, step 402 (or an additional step preceding step 404)comprises outputting the initial set of search results to a user. Forexample, a listing of search results comprising hyperlinks to variousweb sites may be output to a user via a web browser interface.

In step 404, an input is received from the user who requested thesearch, the input indicating at least one undesired search result of theinitial set of search results. Step 404 may, in some embodiments, beperformed after the initial set of search results determined in step 402is output to the user. Thus, the indication received in step 404 maycomprise a selection of a particular one or more search results includedin the initial set of search results along with an indication that theselected one or more particular search results are undesirable. In someembodiments, the selection of a result and the indication that it isundesirable may comprise the same and singular input. For example, aninterface consistent with embodiments within the scope of process 400may be output to the user, the interface allowing the user to check off,select or otherwise indicate which search results of the initial set ofsearch results are undesirable. FIGS. 7 through 14, described below,illustrate several example interfaces which may be output to a user forallowing the user to provide an indication such as may be received instep 404.

In step 406 a characteristic of the one or more search results for whichthe input was received in step 404 is determined. It should be notedthat more than one characteristic may be determined. Such acharacteristic may, in one embodiment, be indicated (e.g., typed in,identified via a code, selected from a menu of available options) by theuser who provided the indication in step 404. Alternatively, thecharacteristic may be determined or inferred on behalf of the user basedon the indication received in step 404 (and, in some embodiments,additional information associated with the user or other users). Ineither case, the characteristic may be utilized in identifying what itis about the one or more search results indicated in step 404 thatrenders them undesirable (and thus allows a removal from the initial setof search results of not only the search result(s) indicated in step 404but also similar, related or relevant search results).

Examples of characteristics that may be indicated, inferred or otherwisedetermined include, without limitation:

(i) a source of the content or material comprising the search result(e.g., considered reliable or unreliable by the user or other users,Society web sites (like American Medical Association, ASPCA, AARP, etc),Government sources, Library of Congress), specific magazine, mediachannel or other content provider;

(ii) the nature, format or medium of the search results (e.g., whetherit comprises a video, one or more photos, a news articles of a web basedpublication or a news articles of a radio/TV or other mediapublication);

(iii) country of origin or other geographical location associated withthe search result;

(iv) type face or other aesthetic characteristic;

(v) language;

(vi) author or publisher;

(vii) the fact that it is a translation or version of same documentalready included as another search result;

(viii) peer review journals;

(ix) scientific;

(x) liberal arts (any category, e.g. philosophy, music, humanities,classics, Greek mythology, roman history, etc);

(xi) time period (e.g. before 1920, after 1800, BCE, future, past,prehistoric, inclusive dates 1969-1972, exclusive dates, not 1980 to1990, or by decade, century or millennium)

(xii) a characteristic of the author such as it being a single Author,multiple author, authors only with multiple publications on same orsimilar topic, professors, teacher of a certain institution or group ofinstitutions (e.g., Oxford, an Ivy League school, the Big ten, etc.);

(xiii) the fact that the search result comprises a PhD thesis;

(xiv) the fact that the search result comprises a document orinformation about an IPO (Initial public offerings);

(xv) the fact that the search result comprises an investment report;

(xvi) the fact that the search result comprises a text book or a giventext book or a text books only on a specific topic;

(xvii) the fact that the search result comprises a lecture;

(xviii) the fact that the search result comprises an advertisement; or

(xix) the fact that the search result comprises theater orentertainment.

Although process 400 is illustrated as determining a characteristic of asearch result indicated by the user as being undesirable, in otherembodiments the process 400 (or another similar process consistent withembodiments described herein), it should be noted that the determinationof the characteristic is merely one example of determining a reason forwhy a user considers one or more particular search results undesirable.Other mechanisms for determining a reason for why a user considers oneor more search results undesirable may be utilized within the scope ofthe embodiments described herein. In some embodiments, the serverperforming the process 400 may be programmed with an algorithm enablingit to identify a characteristic of the search result indicated asundesirable or to determine a reason why a search results is (or islikely to have been) considered undesirable by a user. In someembodiments, a user may be queried as to the reason he or she considersthe search result indicated in step 404 as undesirable to beundesirable.

In accordance with one embodiment, the user may be provided with anindication or flag as to which search results were selected or preferredby other users (e.g., users within a designated group, location, set oflocations, with one or more shared characteristics and/or within aclass). This may be desirable in order to allow each member of a groupthe option to remove those results in an effort to find new or differentrelevant material. In some embodiments the ability to flag, identify orknow other users search efforts or results can be used to emulate aphysical library by monitoring or reducing access to popular informationin order to force or direct deeper searches into the area of research. Amore detailed discussion of processes addressing an “electronic libraryeffect” is provided below with reference to FIG. 6.

Referring now to step 408, additional search results in the first set ofsearch results that correspond to the characteristic determined in step406 are identified. For example, a the remainder of the first set ofsearch results may be analyzed to identify any additional search resultsthat have, exhibit or are associated with the characteristic.

In step 410, the additional search result(s) identified are removed fromthe initial set of search results, as is the search results indicated instep 404 by the user as being undesirable. Accordingly, the initial setof search results is shortened such that it no longer includes thesearch result indicated by the user in step 404 or any additional searchresults identified in step 408.

Removing a search result from a listing of search results may, in someembodiments, comprise refreshing the listing or outputting a modifiedlisting such that the removed search result no longer appears on thelist. In some embodiments, although the removed search result itself(e.g., the hyperlink to a web page) no longer appears on the modifiedlisting, an indication thereof may appear (and, in accordance with someembodiments, such an indication may be usable by the user to add theremoved search result back to the listing of search results). In otherembodiments, removing a search result may comprise hiding the searchresult, graying out the search result, making the search result aninactive hyperlink (in embodiments in which search results comprisehyperlinks) or otherwise altering an indicator of the search result suchthat it is clear to the user that the search result has been removed. Instill other embodiments, removing a search result may comprisereordering or re-ranking the search results such that the undesirablesearch results are moved to the end of a listing of the search results.

In step 412, a modified set of search results is output to the user. Themodified set being the initial set of step 402 minus the search resultindicated in step 404 and minus any additional search results identifiedin step 408. In some embodiments, any removed search results may nolonger appear on a listing of the search results comprising the modifiedset of search results. In other embodiments, the modified set of searchresults includes some indicator of the removed search results such thatthe user is able to add the removed search results back into the listing(e.g., by clicking on the indicator or otherwise utilizing a providedmechanism for requesting that the removed search result be added backinto the listing). In some embodiments, any removed search results maybe output to the user in a separate and distinct listing (e.g., adistinct web page may be created and output to the user, perhaps as aseparate tab, via an interface of a web browser). In some embodiments,both the search result removed as a direct input from the user that thesearch result is considered undesirable is output as well as a listingof any additional search results removed as a result of step 408.

It should be noted that process 400 may be a dynamically appliedprocess. In other words, each time a user indicates a particular searchresult to be undesirable (e.g., a user begins reviewing a list ofinitial search results and starts checking off search results the userfinds undesirable as he or she goes along the list of search results),steps 406, 408 and 410 may be performed such that the list of searchresults is updated or refreshed dynamically as the user provides thenegative feedback (e.g., the search result listing is regenerated toreflect the removal of the undesirable search results). Further, in someembodiments a user may be provided with an “undo” option such that theuser may bring a search result (and any associated or correspondingsearch results which were removed as a result of the user previouslyindicating the search result as being undesirable) if the user changeshis or her mind. For example, an interface of search results previouslyremoved in the search and/or search results previously indicated by theuser in the search as undesirable may be output to the user. Inaccordance with some embodiments, at any time during the process 400 oranother process for allowing a user to indicate undesirable searchresults and have a search result listing modified as a result, if theuser sees a desired result, the user can opt to see that result by anyof the methods available for redirecting a search result to a new page(e.g., clicking, touching, mouse over or other method of pageactivation) or continue to be offered user directed feedback options(e.g., for indicating undesirable search results).

In an embodiment in which a list or set of search results is updateddynamically based on user feedback, the process 400 may loop back tostep 404 upon outputting the modified set of search results, todetermine whether another input has been received from the user,indicating another undesirable search result. If so, the process 400 maycontinue to steps 406, 408 and 410 as described above. In someembodiments, if an user indicates an additional search results as beingundesirable after having already previously indicated in the searchanother search result as being undesirable, the process 400 may includea query step to determine whether the additional search result indicatedby the user as undesirable has already been removed from the initial setof search results (e.g., because it corresponds to the characteristicidentified in step 406 and the modified set of search results had notyet been output to the user prior to the user indicating the additionalsearch result as being undesirable).

In some embodiments, a search listing or set of search results is onlymodified upon the user's request to so modify the set of search resultsby applying any negative or positive feedback (e.g., in the form ofspecific search results the user considers undesirable). In suchembodiments if applied to the process 400, the steps 404-412 may only beinitiated or launched upon some activation or request by the user suchthat multiple search results flagged or identified by the user asundesirable (e.g., each associated with a distinct characteristic orreason for being considered undesirable) are considered and applied inone application of the process and the user is provided with an outputof modified search results which reflects the removal of all the searchresults (i) identified by the user as undesirable; and (ii) identifiedby the server or software application as being undesirable based on theuser's indication of undesirable search results. In some embodiments, asdescribed herein, the feature or service of modifying search results toremove undesirable search results may be embodied as a softwareapplication that is overlaid onto or run in conjunction with a searchengine but that is distinct from the search engine's functionality ofrunning an algorithm to search through available data for searchresults. In such embodiments which encompass the functionality ofrunning process 400 only upon an initiation of a user (e.g., when theuser determines in his discretion that he is sufficiently done withselecting undesirable search results and would like the search resultlisting modified to reflect his selections), the modification of thelisting of search results may be initiated upon the user launching thesoftware application or the relevant feature thereof.

It should be noted that the above-described mechanism allows for batchor group application of the user directed feedback. The same informationcould be provided to a designated user associated with a group (e.g.,group monitor, leader, teacher, research director or other similarperson) so the group work could be monitored. The users and/or monitorof the group work can use this information to reduce duplication of workor better understand the direction the project is taking. In accordancewith some embodiments, the application of the new information may betracked as the user indicates the modifications and these modificationscan be dynamically undone by a specific activation by the user. A moredetailed description of embodiments directed to group research projectsis provided below, with respect to the “electronic library effect”embodiments.

The method of step 400 is one example of how an overwhelmingly large setof search results may be shortened to a more digestible number of searchresults based on a user's indication of one or more search results thatthe user quickly identifies as undesirable. Of course, other methods maybe utilized to achieve a similar result. For example, rather thandetermining a characteristic of the search result indicated by the useras being undesirable and finding additional search results thatcorrespond to or share the characteristic, a server or softwareapplication may determine a search result the user indicates isundesirable and remove additional search results that are otherwiseassociated with that search result. For example, the server or softwareapplication may remove additional results that were similarly indexed ina database searched in identifying the initial set of search results orthat are somehow stored as related search results in the database.

In some embodiments, when a user provides feedback regarding a listingof search results (e.g., a user indicates one or more particular searchresults in the listing are considered undesirable), such an indicationfrom the user may be fed back to the search tool and the search may bererun using this user provided information. In other words, although insome embodiments a user's indication that one or more search results areundesirable may simply cause a removal of the undesirable search results(search results affirmatively indicated by the user as undesirable aswell as search results determined to be undesirable based on the user'sindication of the one or more particular search results) in otherembodiments the user's indication that one or more search results areconsidered undesirable may cause a new search to be performed based onthis indication(s). For example, if the user is providing theindication(s) of the one or more undesirable search results directly toa search tool, the search tool may initiate a new search of the samedata (e.g., proprietary database of the search tool) previously searchedupon receiving the indication(s) of the undesirable search results. Inanother example, if the user is providing the indication(s) of the oneor more undesirable search results to a third party service or softwareapplication which utilizes the search results returned by a search tool,such third party service or software application may communicate withthe search tool to request a new search based on the indication(s) fromthe user (e.g., the third party service or software application maygenerate one or more additional search terms or rules for the searchtool to use in performing a new search).

Turning now to FIG. 5, illustrated therein is yet another process formodifying a set of search results to remove undesirable search resultsand thus increase the likelihood of serendipity. More specifically,illustrated therein is a flowchart of a process 500 consistent with someembodiments described herein. It should be noted that process 500 isexemplary only and should not be construed in a limiting fashion. Forexample, additional and/or substitute steps to those illustrated may bepracticed within the scope of the present invention and in one or moreembodiments one or more steps may be omitted or modified. In oneembodiment, the process 500 is performed by a search tool server 112,114 or 116. Alternatively, process 500 may be performed by a third partyserver 118 and/or a software application residing on or otherwiseutilized by (e.g., using cloud storage technology) a user device of auser requesting the search.

An initial set of search results is output to a user in step 502. Thisstep may be similar to step 402 of process 400. Accordingly, step 502may include and any additional or preceding processes or steps, such asreceiving search terms and an initiation of search from a user as wasdescribed with respect to step 402. As indicated in step 402, in someembodiments determining an initial set of search results may alsocomprise outputting the initial set of search results to a user.However, it is contemplated that in many (but not all) embodimentswithin the scope of process 500, the initial set of search resultsdetermined in step 502 is not output to the user who requested thesearch. Rather, the initial set of search results may be pared down forthe user before it is output in modified form, in accordance with thefollowing steps.

In step 504 a user-selected rule for defining undesired search resultsis determined. For example, upon requesting a search and providingsearch terms to be used in conducting the search, the user may haveselected or indicated one or more rules defining results or types ofresults the user considers undesirable. The user may have provided suchan indication or selection by typing in information, making a selectionfrom a menu of available options, or a combination thereof. For example,in some embodiments a user may be interested in viewing search resultsthat are not the most popular search results (e.g., results that are notranked the most highly by the search engine performing the search) orthat are not ranked highly due to payments provided by a source of thesearch results (some search engines allow search result sources to payin order to have their search results placed higher in a listing ofsearch results for certain search terms). In one embodiment, a user maybe provided with a menu (e.g., by a search engine provider or by a thirdparty service provider who modifies a set an initial set of searchresults returned by a search engine) of available rules for definingundesirable search results. Examples of such rules include, withoutlimitation: (i) do not output the top X most popular search results;(ii) do not output search results from source X; (iii) do not outputsearch results associated with source X; and (iv) do not output searchresult which also include term X. It should be noted that the “X” termof each rule may, in some embodiments, be a “fill in the blank” type ofterm that the user can specify or customize by filling in a value for itor selecting one of a plurality of available values. In otherembodiments the “X” term may already be specified and not customizableby the user. In some embodiments, a database of rules from which a usermay select a rule for defining undesired search results may be stored(e.g., in a memory of a search engine server) and a user-friendlydescription or indication of each such available rule may be may be madeavailable to any user beginning a search.

Step 504 of determining a user-selected rule may comprise identifyingany rule (and, in some embodiment, particular value for a parameter ofthe rule that is available for customization to the user) as it had beenpreviously selected by the user upon requesting the search. For example,a user may be allowed to select such a rule at the time of inputting theone or more search terms for a search. Thus, such a rule may be storedin a memory (e.g., a temporary memory of the user device, a searchengine server, or another memory accessible to the search engine server,third party server or software application facilitating the search). Inother embodiments, a user may be prompted to select a rule (if any) fordefining undesired search results at another time (e.g., while thesearch is being performed; after the search is performed and an initialset of search results is identified, either before or after the initialset of search results it output to the user). In some embodiments, morethan one rule may be selected by the user and determined in step 504. Insome embodiments, one or more rules may be stored in association with auser profile and used for more than one search (e.g., the user mayselect such a rule and have it applied to more than one search, suchthat the user-selected rule is not necessarily specified by the user forthe particular search whose search results are being modified in process500).

In accordance with some embodiments, the initial set of search resultsmay indeed be output to a user in step 502. It may be output along witha suggestion for removing one or more search results, in accordance witha rule utilized by the software application which determines such asuggestion. For example, the top five (5) search results may be flaggedor otherwise indicated to a user as ones the user should considerremoving. If the user accepts the suggestion (e.g., by selecting thesuggested search results for removal or answering affirmatively so aquery such as “Would you like the 5 search results removed from thesearch?”), the user's agreement to remove the search results flagged forpossible removal in accordance with the rule may be considered aselection by the user of the rule. In some embodiments, search resultsmay be suggested for removal based on a particular user's (or otherusers) selections of search results either for removal or furtherinvestigation, in previous searches. For example, if the user neverclicks on search results comprising hyperlinks to foreign languagesites, any search results which include foreign language content may besuggested for removal.

Returning now to process 500, in step 506 the search results whichsatisfy the rule determined in step 504 are identified. For example, ifthe user selected a rule which defines the top 5 most popular searchresults as undesirable, the top 5 most popular search results may beidentified in step 506. The search results identified in step 504 assatisfying the rule are then removed from the initial set of searchresults in step 508. Removal of a search result from an initial list ofsearch results may be carried out in a manner similar to that describedwith respect to process 400 (FIG. 4).

In step 510, a modified set of search results (one which reflects theremoval of the search results performed in step 508) is output to theuser. As with process 400, the outputting of a modified set of searchresults may be accompanied by an outputting of an indication (e.g.,either on the same page or tab as the modified set of search results oron a different page or tab of a web browser) of the removed searchresults. A user may thus be provided with a mechanism to add one or moreremoved search results back into a listing of search results. Further,in some embodiments, if a user is not satisfied with what is output asthe modified set of search results the user may be allowed to modify orchange the rule(s) (and/or value(s) of any customizable parameter(s)thereof) used to identify and remove search results in steps 506 and504. If the user does so modify a rule, the process 500 may be rerunbeginning at step 504 in accordance with the modified or new rule(s) orvalue(s).

It should be appreciated that both process 400 and process 500 allows auser to efficiently remove many undesirable results from a listing ofsearch results without needing to go through an arduous andtime-consuming process of removing each undesirable search result on anindividual basis. A solution which would require the user to indicateeach particular search result that is undesirable and simply remove thesearch result the user affirmatively selects as undesirable necessarilyassumes the user will review each search result. Given the breathtakingmagnitude of search results many searches return (e.g., as illustratedin the example search results of FIGS. 3A-3E), it is unrealistic toassume a user would have the time or patience for such a task. Asdescribed above, a user typically becomes frustrated when faced with alarge number of search results and it is the rare user who has the timeto spend reviewing each search result one-by-one. Solutions such asthose illustrated in process 400 and process 500 allow a user to providefeedback on a few search results (selecting a few search results asbeing undesirable, based on, for example, a quick perusal of searchresults and flagging those that quickly jump out at the user as beingundesirable), or simply provide a characteristic, rule or otherindicator defining an undesirable search result and thus initiate aprocess which removes (or determines whether it is appropriate toremove) additional search results from a listing of search results basedon the user's limited negative feedback and review. For example, assumea particular search returns 1.2 million search results and a userreviews the first two pages of the search results, with twenty (20)search results being listed on each page. Further assume that the userquickly identifies six (6) of the forty (40) search results as beingundesirable (e.g., in accordance with process 400). An algorithm runningin accordance with process 400 or a similar process may thus review theadditional 1.1+ million search results and identify an additional150,000 search results that are also undesirable based on the user'sidentification of the six particular search results. That is,significantly, 150,000 search results that may be removed from a listingof the search results that the user never had to review or flag asundesirable. The removal of these search results may significantlyincrease the user's chances of identifying a search result the user waspreviously unaware of and may have missed if the 150,000 “noise” searchresults had not been removed for the user.

It should be understood that processes 400 and 500 are merely examplesof processes which may be implemented to effectuate some embodimentsdescribed herein. Described herein are various embodiments forstrategies, processes, systems and interfaces which are operable toallow a search tool user better access and control over search queriesand the results. The interfaces, processes and systems described herein,and the associated functionalities are search tool independent.Described herein are various examples for orders of operations search,applications of search modifying options over the current searchresults, options selections, application of modifications, the abilityto “undo” or “redo” an applied option and the ability to repeat thesesets in an iterative fashion as well as alternative ordering of the samefunctional operations. In accordance with some embodiments, a user mayinput a query through any acceptable interface, the search engine or athird party service presents the search results and the user ispresented with direct feedback options for the search results. A goal ofat least some embodiments described herein is to provide systems,methods and interfaces operable to aid a user of a search tool to removeunwanted results or “noise” efficiently as opposed to enhancing,recording or using positive reinforcement as the only method ofobtaining a better result. The search result modification optionsdescribed herein comprise an effective means to reduce the noise andbring the more pertinent results closer to the top of the list whilereducing the list sufficiently enough to allow for serendipity. The usercould then view more pertinent knowledge and see information that theywere unaware of prior to the application of the user modifications. Thedynamically applied negative as well as potential positive feedbackcreated by the user feedback interfaces described herein may, in someembodiments, (i) be stored to help future user searches, and/or (ii) betracked by a computer's location (URL) or a user login to better assistin future searches (of the same user and/or different users). Inaccordance with some embodiments, one or more of the user directedmodifications can be added to the engines data base as it improves itsown methods of searches in order to reduce “noise” or unwanted data andimprove serendipity with the goal of a better user experience.

Another area where the current state of the art fails to provide truedepth of knowledge is what is referred to herein as “the electroniclibrary effect.” The problem addressed by the group research embodimentsdescribed herein has been recognized (but not solved) on collegecampuses and highlighted now in an article printed in the University ofPennsylvania's newspaper, The Weekly Pennsylvanian, on Apr. 3, 2011. Thearticle entitled “How the Internet Killed the Creativity” with thesubtitle “Relying too much on search engines limits, rather than helps,our search abilities.” the author, Sarah Banks, has further emphasizedthis unforeseen and unsolved problem. The article quotes also NicholasCarr's article in the Atlantic magazine in 2008 on how skimminginformation on the web as opposed to deeper learning is making us allless intelligent. To make her point, Ms. Banks notes that web basedresearch is the norm on college campuses and this has had an unexpectednegative effect on the papers produced by the students. She noted thatwhen a paper on a given topic is assigned to a class, the students godirectly to the web. They quickly search the topic and the modern searchengines direct them to the most linked or most popular pages that matchtheir query. The outcome of the simple application of being too good atproviding these results in short order is that each student gets thesame references, gains rapid access to those references and hence theyall write similar papers. This is different from plagiarism but lookssimilar in outcome. If the students copied each others' work there wouldbe an outrage but if the web's engines and algorithms corral them intosimilar results that is not fully recognized yet as a problem inlearning that is need of a solution. Ms. Banks' article is evidence thateven the students themselves are beginning to recognize the limits ofsearch engines and the pitfalls within the current state of the art.This is a relatively new phenomenon and is what one could term “theelectronic library effect.”

“The electronic library effect” was not seen in the prior era of thephysical library nor is it the expected outcome in the modern electroniclibrary (the web) based on having the terabytes of knowledge at thestudents' fingertips. If we ask why this phenomenon did not happen inthe conventional library system, we are pointed to two factors. First,the physical requirement of being able to take a book out and read itmeant that not all the students can have the same book at the same time.The access to the same materials at the same time is entirely possible,if not extremely likely, on the web. In fact, this effect according theBanks article based on the University of Pennsylvania experience hasbecome the rule not the exception. In contrast, in an old fashionedphysical library, faced with the challenge of not being able to get thebook one wanted from the library the students were forced to look at awider group of materials. They may have had to thumb through a number ofbooks and articles to find something of value to write about. In thephysical library, serendipity was the rule not the exception. Withlimited access to the most popular materials, there was no “electroniclibrary effect.” The result was a wide variety of papers from oneassignment. Here again is the search engine paradox. With moreinformation available to all, there is less variety in the discoveriesmade by a group of independent researchers.

Applicant has identified a solution to the “electronic library effect”in some of the embodiments described herein. Applicant has recognizedthat there is new and unique value in the ability for researchers toaffirmatively remove a very popular result (and all similar results)from a search or from a listing of search results output to researchesfor a given research project. The idea that removing a popular referenceor result from a search to create a better if not a more interestingresult while increasing serendipity is counter intuitive. Here theability to negatively influence a search result by removing popularinformation actually has a net positive benefit to the user. Thisunexpected result, especially given the power of the current searchengines, is something that is not obvious to the current users ordesigners of these engines and their associated algorithms. Anelectronic search strategy that allowed groups to be aware of, or limit,the simultaneous use of materials could, again in a counter intuitiveway, decrease duplication of work, reduce the “electronic libraryeffect” and improve the collective research effort.

At least some embodiments described herein help reduce the “electroniclibrary effect” by allowing the user to remove a popular result from asearch with the same technology and ease used to remove the unwantedresults as described above with reference to individual searches. Such afeature would force a search tool to look deeper into a topic forrelevance and a user would experience more serendipity by making thischoice.

Accordingly, some embodiments create the ability, in a class or group ofuser such as students in a given class, users associated with a givenlocation, or other defined group of users, to share in the process ofselecting and deselecting one or more search results from a listing ofresult results. This would increase the chance that each student coulddiscover something new. This would in turn reduce the “electroniclibrary effect” and increase access to and the benefit of the terabytesof data on the web that current search engines are unintentionallymaking less available to the users today.

In accordance with one embodiment, a search engine or another component(software or hardware) operable to communicate with the search enginemay be operable to track and/or record an identified user group (forexample a class taking a given course or working on a particularassigned research project) and inform the students of information suchas how many of their classmate have picked a reference. This allows agiven student, if desired, to remove one or more references from alisting of search results returned to him or her for a search in orderto more easily identify and consider what has not been picked yet. Suchembodiments may create a working environment for teams working on jointresearch projects which allows for a reduction in duplication of effortapplied during the research.

In accordance with some embodiments, the input or feedback that one ormore uses of an associated group of users may provide includes but isnot limiting to (i) the hit or click through count next to each searchfor (within) the designated research group, (ii) an identifier of eachgroup member working on a given search result, and/or (iii) time spendon each result as a method of evaluating the current status of theproject and aid in the selections or time spent on each result as theproject moves forward. For example, in one embodiment a hit countindication may be provided in or next to an area of an interface forallowing the user to indicate which results are unwanted or undesirable(e.g., a red box next to each search results in accordance with oneembodiment). In another example, in one embodiment a time clockindication and/or an indication of initials or researcher code numbermay be similarly output.

In accordance with some embodiments, a professor or other userdesignated as a leader, manager or controller of a group may be providedwith a means of setting a value for one or more parameters defining agroup search project. For example, such a group manager may be providedwith a mechanism for limiting the time spent on a given reference (e.g.,by a single user or by multiple users combined) and/or number of userswho may view or be provided access or a link to a given reference orsearch result. In some embodiments, such a group manager may also beprovided with a means of monitoring a research project and during thecourse of such a research project be allowed to adjust or change a valuefor one or more such parameters.

Referring now to FIG. 6, illustrated therein is a process forfacilitating a research project of a group in a manner that aims tominimize the electronic library effect described herein. Morespecifically, illustrated therein is a flowchart of a process 600consistent with some embodiments described herein. It should be notedthat process 600 is exemplary only and should not be construed in alimiting fashion. For example, additional and/or substitute steps tothose illustrated may be practiced within the scope of the presentinvention and in one or more embodiments one or more steps may beomitted or modified. In one embodiment, the process 500 is performed bya search tool server 112, 114 or 116. Alternatively, process 600 may beperformed by a third party server 118 and/or a software applicationresiding on or otherwise utilized by (e.g., using cloud storagetechnology) a user device of a user requesting the search.

A user identifier of a user initiating a search for a search or researchproject is received in step 602. For example, a username, a user e-mailaddress, a unique user identifier and/or user password may be received.In some embodiments, a research project identifier may also be received.In other embodiments, a research project identifier may be determinedbased on the user identifier received. For example, in some embodiments(whether process 600 is being performed by or in association with asearch tool that is available to the general public (e.g., the GOOGLEsearch engine) or by a private search tool or on a private system ordatabase utilizing the features described herein), a user who is aparticipant or member of a group research project may be able to accessa web page or interface of the search tool that is designed for searchesassociated with registered research projects. Such an interface mayprovide an input mechanism for entering the user identifier and/orproject identifier. A person who is a research project manager may havepreviously registered a research project (and, e.g., selected or beenassigned a unique research project identifier for the project). In someembodiments, the project manager may have input the user identifiers ofall users associated with the project. In other embodiments, associationof a user with a registered research project may be dependent on a userregistering himself with the research project (e.g., users can searchfor and select available projects to join or may have been provided aresearch project identifier and/or password by the project manager inorder to register themselves with the project). However the associationof a user identifier with a research project came about, the useridentifier is utilized to retrieve from a memory (e.g., from a databaseof registered research projects and their associated information) aresearch project for which the user is initiating a search. In otherembodiments, a project identifier is received in step 602 in lieu of auser identifier.

In step 604 a stored search rule associated with the research project isretrieved (e.g., from a record of a database storing information onregistered research projects). The search rule comprises a rule forgoverning an output of search results to one or more users requestingsearches for the research project. For example, a rule may define one ormore search results that are to be removed from a listing of searchresults returned for a search based on search terms provided by theuser, the removal to be applied before the listing is output to theuser. In such embodiments, a such a rule may thus be thought of as arule which defines disallowed search results for a particular researchproject. In another example, a rule may define information that is to beoutput to a user for a given search result returned to the user for asearch conducted in accordance with search terms supplied by the user(e.g., a number of times a particular search result has been output toother users associated with the research project).

In some embodiments, a value may be associated with such a rule. Thevalue may in some embodiments be a fluctuating or dynamic value that isupdated based on searches performed by users associated with theproject. A rule may be selected or provided for association with aparticular research project by the project manager (e.g., a professor ofa course). In other embodiments, one or more users may select or providesuch a rule. More than one such rule may be associated with a givenproject. In some embodiments a rule may be applicable to all usersassociated with a given project. In other embodiments, a rule may beassociated with only a subset (e.g., one) user of a plurality of usersregistered with a project. In some embodiments, a rule may be personalto a particular user such that the rule may not necessarily beassociated with the project but may be associated with a particular userwho is in turn associated with the project.

In one embodiment, a rule comprises at least one characteristic definingat least one search result to be removed from a set of search resultsoutput to a user of the search group other than the manager, such thatany search result having the at least one characteristic is removed froma set of search results output to a user of the search group other thanthe manager. Another example of a rule comprises disallowing searchresult comprising the highest ranked search results (e.g., determined tobe the most relevant to a search by a search tool using the search termssupplied by the user) or most popular search results (e.g., most popularamong the users associated with the particular research project). Yetanother example of a rule comprises defining a maximum number of timesone or more search results is to be included in sets of search resultsfor the search project, such that a particular search result is to beconsidered a disallowed search result once the particular search resultappears in sets of search results for the search project the maximumnumber of times.

As should be understood from considering the above description of rulesfor governing the output of search results, some embodiments provide forthe tracking of sets of search results output to users associated with aparticular research project (e.g., by a search tool, third partyservice). Such information may be utilized to update a value of aparameter of a rule. Additionally, some embodiments provide foroutputting to a user associated with a given research project anindication of information regarding searches (and search results) ofother users associated with the research project. Examples of suchinformation which may be tracked, utilized to update values of rulesand/or output to users include, without limitation, for a given searchresult: (i) a number of instances in which the search result has beenincluded in a set of search results output to the users of the searchgroup; (ii) a duration of time a single user of the search group hasspent reviewing a content of the search result; (iii) a total durationof time all users of the search group who had the search result outputto them in a set of search results reviewing the content of the searchresult (e.g., an average duration based on a total number of users towhom the search result has been output in a set of search results); and(iv) a number of instances in which the search results has been selectedby a user of the search group.

In step 606, at least one search term for a search of data, the searchbeing associated with the research project, is received from the user.It should be noted that in some embodiments step 606 may be performedprior to step 604 (or even prior to step 602).

In step 608 an initial set of search results is determined based on theat least one search term received in step 606. For example, a searchtool may perform a search of a database of indexed information based onthe search term(s). In step 610 it is determined whether any of thesearch results determined in step 608 are impacted by the rule(s)determined in step 604. For example, each search result of the initialset of search results may be compared to the parameters and values ofeach rule associated with the research project to determine whether itsatisfies or falls within the rule. If none of the search results of theinitial set of search results are impacted by the rule (e.g., none ofthe search results are defined as disallowed search results by a ruleand/or there is not tracked information to output to the user for any ofthe search results), the initial set of search results is output to theuser without modification in step 612. Otherwise, the process 600continues to step 614 wherein the initial set of search results ismodified based on the one or more rules impacting it. For example, ifone or more search results are defined as disallowed search results byan applicable rule, such disallowed search results are removed from thelisting of search results. In another example, if there is informationtracked and stored for a given search result (e.g., how many times thesearch result has been output to or selected by other users associatedwith the research project), such information is added to the initial setof search results (e.g., text conveying the information is added to alisting of search results, in association with the relevant searchresult to which it pertains). In step 616 the modified set of searchresults is output to the user.

Process 600 is merely one example process of how embodiments describedherein may be applied to remove selected popular or wanted results (oradd pertinent information regarding the popularity of wanted or popularor wanted results based on other users' activities), at the user's orproject manager's discretion and direction, to reverse the so called“electronic library effect.” In this way, at least some of theembodiments may facilitate a new counter-intuitive strategy or method ofgaining access to less accessible data. One or more embodimentsspecifically simplifies the process of removing results so that thisunique method of indirectly improving search results by removingdesirable results would be available to even the least experiencedresearcher.

For example, as described above, some embodiments would allow, through alogin, location or common set of URLs (for a school, college oruniversity network), a group or class of students to be made aware ofthe results others have used, selected or been made aware of. A groupand a group leader or monitor could be identified via a loginembodiment. The leader or monitor could have access to the group'sactivity and the option to limit access to popular results as a methodto encourage deeper or expanded research. This would have particularvalue to those trying to look deeper into a topic, experience moreserendipity and find less common or more unique points of view. Thestrategies and processes described herein may also be applied to a groupworking on the same project simultaneously in order to reduceduplication of work.

Example Interfaces

Turning now to FIGS. 7 through 14, illustrated therein are variousexample interfaces, input mechanisms and applications useful inconjunction with any search tools (e.g., search engines or algorithms),whether currently available or yet to be designed. The interfaces andits functionalities are search tool independent. The features andfunctionalities described herein can be dynamically applied as the useradds information about the search in the simplest form possible (a clickbox). This provides the user instant feedback on the modifications,creating a new user experience. In accordance with some embodiments(e.g., the embodiment of FIG. 6), the features and functionalitiesdescribed herein may also operate in a way to apply the searchmodifications in a group of users. Some embodiments comprise a simpleinterface for allowing a user to provide feedback on a search result(single click box with dynamic application). Other embodiments providefor a more complex interface for allowing a user to provide feedback ona search result (slide bar valuations and group or batch application ofthe user modifications). The order of operations search, application ofsearch modifying options over the current search results, optionsselection, application of modifications, the ability to “undo” or “redo”an applied option and the ability to repeat these sets of steps in aniterative fashion as well as alternative sequences of the samefunctional operations are contemplated.

In accordance with one embodiment, the user inputs one or more searchterms for a search through an available interface, the search toolpresents the initial results and the user is provided with simple andeasy to use user feedback options. These options are used to betterdefine the search, reduce the unwanted results or “noise” and bring themore pertinent or desired search results closer to the top of the list.A goal is reducing the list sufficiently to allow for serendipity, theviewing of pertinent knowledge the user was unaware existed prior to theapplication of the user modifications. In some embodiments, the negativeas well as potential positive feedback created by this novel userfeedback interface may be stored to help future searches; it may betracked by a computer's location (URL) or a user login to better assistin future searches (of the same user and/or other users). A location orgroup login feature could be used to allow the members of the group tohave knowledge or indication of the other group members activity alongwith the ability to remove others' favored results in the researchprocess to avoid duplication of results with a group or class of uses.The user directed modifications can be added to the search engines andother search tools processes as an improvement of methods of searches inorder to reduce “noise” or unwanted data and improve serendipity withthe goal of a better user experience.

The example embodiments of user interfaces illustrated in FIGS. 7through 14 include one or more mechanisms for allowing a user to provideinput or feedback regarding one or more search results. It should benoted that although the example interfaces and input mechanismsillustrated use graphics, text and other display mechanisms, othermechanisms are contemplated. For example, in one or more embodimentssearch results and/or user input or feedback on one or more searchresults may be exchanged via audio (e.g., a user can speak into amicrophone of a device providing the search results in order to provideinput or feedback on the search results).

Illustrated in the example embodiment of FIG. 7 is single radio button(56) input mechanism associated with each respective search result (54).The user may click the radio button on or off as desired to flag aparticular search result 54 as undesirable (e.g., the search result maybe flagged as undesirable by clicking the radio button on). Illustratedin FIG. 8 is an interface which provides two buttons, 50 and 52, whichmay in some embodiments each be in a different color or other distinctconfiguration, for each search result 54. For example, a green orpositive button (50) may be provided for flagging an associated searchresult as particularly desirable and a red or negative button (52) maybe provided for flagging an associated search result as undesirable. Insome embodiments the input mechanisms 50 and 52 may comprise plus orminus signs or any simple indicator that relates a positive or negativefeedback as the language or culture of the user dictates in order tomake their purpose clear. These may be applied over or next to thesearch results.

It should be understood that embodiments contemplated are not limited toone or two button options or binary user feedback options; in someembodiments a user may be provided with a mechanism of providing inputor feedback on search results that allow the user to select one of anynumber (e.g., a number greater than two) of feedback or input options,ratings or values for a given search result. For example, any number ofbuttons may be defined to add in the search definition. Illustrated inthe embodiment of FIG. 9 is one embodiment in which a variable positiveor negative attribute is applied to each individual search result via aslide bar (62) on a line (64). Then ends are denoted as top (+) (60) andbottom (−) (66) and could be embodied, for example, as color coded(green/red) or symbol driven as the language or culture of the userdictates. In this way the search tool can be supplied with a relativeinput as to the user's valuation of each of the individual searchresults (or for whichever search results the user desires to providefeedback).

Illustrated in the embodiment of FIG. 10 is a check or “x” boxembodiment of the user feedback option with an activation method (70) asshown vi a “go” box as well as a “undo” box (72) for allowing the userto affirmatively direct the application or activation of the searchresult output modification features described herein. In the embodimentof FIG. 10 is also a check box nest to each search result. In oneembodiment, if a user clicks or selects a check box next to a particularsearch result, the user is so indicating that the result is undesirable.If the user leaves the check box blank or unselected, the user indicatesthe associated search result is desirable. Thus, in FIG. 10, the blankor “un ‘x’ed” check box 76 indicates an acceptable result and the checkbox with an X in it (74) indicates an undesirable search result. Thereverse definition of these boxes is contemplated or other symbols (andother words such as apply or remove) as culture and language of the usershould dictate the preferred communication method for this feature.

Illustrated in FIG. 11 is another embodiment in which the highlightedarea (80) indicated, which may comprise a particular search result orportion thereof, may be used by a processor operable to implement one ormore of the functions described herein in a positive, negative orrelative way in order to modify the search by moving an arrow or othericon (82) over the desired action within a pop up type menu. In thisembodiment a mouse click, double tap, multi-finger tap, control click orother differentiating selection on the highlighted area (80) opens adialog box or drop down menu (84) containing options for the evaluationor use of the selected area in the search modification.

The example interfaces illustrated in FIGS. 12-14 are directed to groupresearch embodiments, such as the one described with reference to FIG.6. FIG. 12 illustrates an example embodiment of user directed feedbackoptions to search results in which a user is provided with a mechanismcomprising a single button (90) associated with each search result,which button the user may click to indicate that the associated searchresult is undesirable. Within each button 90 is indicated a hit count(in area 92 of each button) for the associated search result as viewedby the designated user group during a search (e.g., upon review of thegroup manager or as an end of project summary). A hit count maycomprise, for example, a number of times the search result has beenoutput in a listing of search results to any user associated with theparticular research project.

FIG. 13 illustrates another example embodiment in which user directedfeedback options to search results are presented as single buttons whichmay be clicked on to indicate undesirable results. As in FIG. 12, withineach button 90 is the hit count 92 for the search result as viewed bythe designated user group during a search (e.g., upon review of thegroup leader or as an end of project summary). Additionally, illustratedis a mouse-over or pointer element 94 which indicates an identifier ofeach user of the group or associated with the research project who isalso associated with the particular search result (e.g., it may indicatethe initials or user identifier of each user to whom the search resultwas output to who selected the search result for viewing). Such aninterface may allow the users associated with a particular researchproject or a group manager to see the activity associated with aparticular search result, as well as who is working on a particular partof the project at a given time. The identifier(s) indicated in area 94are not limited to individuals; they may comprise identifiers of a classor group or people, a team or location as part of a larger group.

FIG. 14 illustrates another example embodiment in which user directedfeedback options to search results are presented as single buttons whichmay be clicked on to indicate undesirable results. Within each button 90is the hit count 92 for the search result as viewed by the designateduser group as viewed during a search. As in FIG. 13, there is amouse-over or pointer function 94 for indicating user identifiers ofindividual users or groups of users associated with each search result.Additionally, there is provided for each search result an icon orindicator 98 for the total time spent viewing or considering a searchresult (which may serve as an indicator of the usefulness of the searchresult). For example, if a first search result has associated therewitha single hit and 8 minutes spent on it while another search result hasassociated therewith eight hits and only a total of 10 minutes spent onit by all viewers who considered it, this may be valuable information toa user (or project manager) as to the relative value of the informationbeing reviewed by the group). In another example, if 6 people spent 12minutes on one item and another item has 24 hits and a total of 3minutes, it would imply the first reference was found to be morevaluable to the group, information which may be valuable to a userreviewing a listing of search results and attempting to identify themost valuable or pertinent search results.

Of course, other interfaces for allowing a user to indicate one or moreresults as undesirable are within the scope of the embodiments describedherein and the examples described are intended to be illustrative andnon-limiting. For example, a “finger swipe” interface may be utilized inwhich a user “swipes away” a result he considers undesirable by usinghis finger to move it to the left or right (off the screen or onto aside bar of a screen).

Rules of Interpretation

Numerous embodiments have been described, and are presented forillustrative purposes only. The described embodiments are not intendedto be limiting in any sense. The invention is widely applicable tonumerous embodiments, as is readily apparent from the disclosure herein.These embodiments are described in sufficient detail to enable thoseskilled in the art to practice the invention, and it is to be understoodthat other embodiments may be utilized and that structural, logical,software, electrical and other changes may be made without departingfrom the scope of the present invention. Accordingly, those skilled inthe art will recognize that the present invention may be practiced withvarious modifications and alterations. Although particular features ofthe present invention may be described with reference to one or moreparticular embodiments or figures that form a part of the presentdisclosure, and in which are shown, by way of illustration, specificembodiments of the invention, it should be understood that such featuresare not limited to usage in the one or more particular embodiments orfigures with reference to which they are described. The presentdisclosure is thus neither a literal description of all embodiments ofthe invention nor a listing of features of the invention that must bepresent in all embodiments.

The terms “an embodiment”, “embodiment”, “embodiments”, “theembodiment”, “the embodiments”, “an embodiment”, “some embodiments”, “anexample embodiment”, “at least one embodiment”, “one or moreembodiments” and “one embodiment” mean “one or more (but not necessarilyall) embodiments of the present invention(s)” unless expressly specifiedotherwise. The terms “including”, “comprising” and variations thereofmean “including but not limited to”, unless expressly specifiedotherwise.

The term “consisting of” and variations thereof mean “including andlimited to”, unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all of theitems are mutually exclusive. The enumerated listing of items does notimply that any or all of the items are collectively exhaustive ofanything, unless expressly specified otherwise. The enumerated listingof items does not imply that the items are ordered in any manneraccording to the order in which they are enumerated.

The term “comprising at least one of” followed by a listing of itemsdoes not imply that a component or subcomponent from each item in thelist is required. Rather, it means that one or more of the items listedmay comprise the item specified. For example, if it is said “wherein Acomprises at least one of: a, b and c” it is meant that (i) A maycomprise a, (ii) A may comprise b, (iii) A may comprise c, (iv) A maycomprise a and b, (v) A may comprise a and c, (vi) A may comprise b andc, or (vii) A may comprise a, b and c.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

The term “based on” means “based at least on”, unless expresslyspecified otherwise.

The methods described herein (regardless of whether they are referred toas methods, processes, algorithms, calculations, and the like)inherently include one or more steps. Therefore, all references to a“step” or “steps” of such a method have antecedent basis in the mererecitation of the term ‘method’ or a like term. Accordingly, anyreference in a claim to a ‘step’ or ‘steps’ of a method is deemed tohave sufficient antecedent basis.

Headings of sections provided in this document and the title are forconvenience only, and are not to be taken as limiting the disclosure inany way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or moreintermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required, orthat each of the disclosed components must communicate with every othercomponent. On the contrary a variety of optional components aredescribed to illustrate the wide variety of possible embodiments of thepresent invention.

Further, although process steps, method steps, algorithms or the likemay be described in a sequential order, such processes, methods andalgorithms may be configured to work in alternate orders. In otherwords, any sequence or order of steps that may be described in thisdocument does not, in and of itself, indicate a requirement that thesteps be performed in that order. The steps of processes describedherein may be performed in any order practical. Further, some steps maybe performed simultaneously despite being described or implied asoccurring non-simultaneously (e.g., because one step is described afterthe other step). Moreover, the illustration of a process by itsdepiction in a drawing does not imply that the illustrated process isexclusive of other variations and modifications thereto, does not implythat the illustrated process or any of its steps are necessary to theinvention, and does not imply that the illustrated process is preferred.

It will be readily apparent that the various methods and algorithmsdescribed herein may be implemented by, e.g., appropriately programmedgeneral purpose computers and computing devices. Typically a processor(e.g., a microprocessor or controller device) will receive instructionsfrom a memory or like storage device, and execute those instructions,thereby performing a process defined by those instructions. Further,programs that implement such methods and algorithms may be stored andtransmitted using a variety of known media.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle.

The functionality and/or the features of a device may be alternativelyembodied by one or more other devices which are not explicitly describedas having such functionality/features. Thus, other embodiments of thepresent invention need not include the device itself.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing data (e.g., instructions) that may beread by a computer, a processor or a like device. Such a medium may takemany forms, including but not limited to, non-volatile media, volatilemedia, and transmission media. Non-volatile media include, for example,optical or magnetic disks and other persistent memory. Volatile mediamay include dynamic random access memory (DRAM), which typicallyconstitutes the main memory. Transmission media may include coaxialcables, copper wire and fiber optics, including the wires or otherpathways that comprise a system bus coupled to the processor.Transmission media may include or convey acoustic waves, light waves andelectromagnetic emissions, such as those generated during radiofrequency (RF) and infrared (IR) data communications. Common forms ofcomputer-readable media include, for example, a floppy disk, a flexibledisk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM,DVD, any other optical medium, punch cards, paper tape, any otherphysical medium with patterns of holes, a RAM, a PROM, an EPROM, aFLASH-EEPROM, any other memory chip or cartridge, a carrier wave asdescribed hereinafter, or any other medium from which a computer canread.

Various forms of computer readable media may be involved in carryingsequences of instructions to a processor. For example, sequences ofinstruction (i) may be delivered from RAM to a processor, (ii) may becarried over a wireless transmission medium, and/or (iii) may beformatted according to numerous formats, standards or protocols, such asTransmission Control Protocol, Internet Protocol (TCP/IP), Wi-Fi,Bluetooth, TDMA, CDMA, and 3G.

Where databases are described, it will be understood by one of ordinaryskill in the art that (i) alternative database structures to thosedescribed may be readily employed, and (ii) other memory structuresbesides databases may be readily employed. Any schematic illustrationsand accompanying descriptions of any sample databases presented hereinare illustrative arrangements for stored representations of information.Any number of other arrangements may be employed besides those suggestedby the tables shown. Similarly, any illustrated entries of the databasesrepresent exemplary information only; those skilled in the art willunderstand that the number and content of the entries can be differentfrom those illustrated herein. Further, despite any depiction of thedatabases as tables, other formats (including relational databases,object-based models and/or distributed databases) could be used to storeand manipulate the data types described herein. Likewise, object methodsor behaviors of a database can be used to implement the processes of thepresent invention. In addition, the databases may, in a known manner, bestored locally or remotely from a device that accesses data in such adatabase.

For example, as an example alternative to a database structure forstoring information, a hierarchical electronic file folder structure maybe used. A program may then be used to access the appropriateinformation in an appropriate file folder in the hierarchy based on afile path named in the program.

It should also be understood that, to the extent that any term recitedin the claims is referred to elsewhere in this document in a mannerconsistent with a single meaning, that is done for the sake of clarityonly, and it is not intended that any such term be so restricted, byimplication or otherwise, to that single meaning

In a claim, a limitation of the claim which includes the phrase “meansfor” or the phrase “step for” means that 35 U.S.C. §112, paragraph 6,applies to that limitation.

In a claim, a limitation of the claim which does not include the phrase“means for” or the phrase “step for” means that 35 U.S.C. §112,paragraph 6 does not apply to that limitation, regardless of whetherthat limitation recites a function without recitation of structure,material or acts for performing that function. For example, in a claim,the mere use of the phrase “step of” or the phrase “steps of” inreferring to one or more steps of the claim or of another claim does notmean that 35 U.S.C. §112, paragraph 6, applies to that step(s).

With respect to a means or a step for performing a specified function inaccordance with 35 U.S.C. §112, paragraph 6, the correspondingstructure, material or acts described in the specification, andequivalents thereof, may perform additional functions as well as thespecified function.

Computers, processors, computing devices and like products arestructures that can perform a wide variety of functions. Such productscan be operable to perform a specified function by executing one or moreprograms, such as a program stored in a memory device of that product orin a memory device which that product accesses. Unless expresslyspecified otherwise, such a program need not be based on any particularalgorithm, such as any particular algorithm that might be disclosed inthe present application. It is well known to one of ordinary skill inthe art that a specified function may be implemented via differentalgorithms, and any of a number of different algorithms would be a meredesign choice for carrying out the specified function.

Therefore, with respect to a means or a step for performing a specifiedfunction in accordance with 35 U.S.C. §112, paragraph 6, structurecorresponding to a specified function includes any product programmed toperform the specified function. Such structure includes programmedproducts which perform the function, regardless of whether such productis programmed with (i) a disclosed algorithm for performing thefunction, (ii) an algorithm that is similar to a disclosed algorithm, or(iii) a different algorithm for performing the function.

CONCLUSION

While various embodiments have been described herein, it should beunderstood that the scope of the present invention is not limited to theparticular embodiments explicitly described. Many other variations andembodiments would be understood by one of ordinary skill in the art uponreading the present description.

1. A computer-implemented method comprising: determining, by a processorof a computing device operable to modify search results, a firstplurality of search results of a search performed by a search tool, thesearch based on a first instance of search terms entered by a userassociated with the search; receiving, by the processor, an input of theuser, the input indicating at least one result of the first plurality ofsearch results to be removed from the first plurality of search results,thereby receiving an indication of at least one first currentlyundesired search result of the first plurality of search results;removing the currently undesired search result from the first pluralityof search results; determining, by the processor, a remainder of thefirst plurality of search results to be the remaining search results ofthe first plurality of search results after the first undesired searchresult has been removed; identifying, dynamically for the search by theprocessor and based on the input indicating the at least one result, acharacteristic of the at least one first undesired search result byidentifying a plurality of characteristics corresponding to the firstundesired search result and selecting the characteristic from theplurality of characteristics, wherein identifying dynamically comprisesidentifying based on the first instance of search terms entered by theuser and without the search being rerun after the input is received;analyzing, dynamically for the search and by the processor, theremainder of the first plurality of search results to determine whetherany search results of the first plurality of search results correspondto the characteristic; determining, dynamically and by the processor, atleast one search result of the remainder of the first plurality ofsearch results that corresponds to the characteristic to be an at leastone second undesired search result; removing, dynamically and by theprocessor, the at least one second undesired search result from thefirst plurality of search results, thereby determining a secondplurality of search results; and causing the second plurality of searchresults to be output.
 2. The computer-implemented method of claim 1,wherein receiving an input of a user comprises receiving, by theprocessor, an indication of the characteristic; wherein thecharacteristic comprises a first popularity ranking of the at least oneundesired search result; wherein analyzing comprises analyzing, by theprocessor, the remainder of the first plurality of search results todetermine any search results that correspond to the first popularityranking or another popularity ranking which indicates a higherpopularity than that indicated by the first popularity ranking, whereindetermining at least one search result of the remainder of the firstplurality of search results that corresponds to the characteristiccomprises determining at least one search result of the remainder of thefirst plurality of search results that is at least as popular as thefirst undesired search result to be a second undesired search result. 3.The computer-implemented method of claim 1, wherein determining at leastone search result of the remainder of the first plurality of searchresults that corresponds to the characteristic comprises determining atleast one search result of the remainder of the first plurality ofsearch results that is defined by the characteristic.
 4. Thecomputer-implemented method of claim 1, wherein the first plurality ofsearch results comprise results of a search requested by the user. 5.The computer-implemented method of claim 1, wherein the input comprisesa selection of an indicator corresponding to the first undesired searchresult.
 6. The computer-implemented method of claim 1, whereinidentifying the characteristic comprises receiving an indication of thecharacteristic from the user.
 7. The computer-implemented method ofclaim 1, further comprising: outputting to the user the plurality ofcharacteristics; and wherein selecting comprises selecting one of theplurality of characteristics based on a response from the user to theplurality of characteristics output to the user.
 8. Thecomputer-implemented method of claim 1, wherein the characteristiccomprises at least one of: a source of the first undesired searchresult; a ranking by a search tool of the first undesired search result;a popularity of the first undesired search result among one or moresearch tools using one or more specified search terms to perform asearch; a website associated with the first undesired search result; anature of the first undesired search result; an aesthetic characteristicof the first undesired search result; a language in which the firstundesired search result is provided; a geographic location associatedwith the first undesired search result; an author of the first undesiredsearch result; a publisher of the first undesired search result; a timeperiod associated with the first undesired search result; a dateassociated with the first undesired search result; and text associatedwith the first undesired search result.
 9. The computer-implementedmethod of claim 1, further comprising: receiving from the user, prior todetermining the first plurality of search results, a request for thesearch, the request defining at least one search term; and performingthe search in accordance with a search algorithm using the at least onesearch term.
 10. The computer-implemented method of claim 9, furthercomprising: performing, after determining the first undesired searchresult, the search again using the search algorithm and the indicationof the at least one undesired search result.
 11. Thecomputer-implemented method of claim 9, further comprising: performing,after determining the remainder of the first plurality of searchresults, the search again to identify additional search results thatshare a characteristic of at least one result of the remainder of thefirst plurality of search results.
 12. The computer-implemented methodof claim 1, further comprising: transmitting, to a search tool, anindication of the at least one undesired search result along with arequest to renew the search based on the indication of the at least oneundesired search result.
 13. The computer-implemented method of claim12, wherein transmitting the indication of the at least one undesiredsearch result comprises transmitting an indication of the characteristicof the at least one undesired search result.
 14. Thecomputer-implemented method of claim 1, further comprising:transmitting, to a search tool, an indication of the remainder of thefirst plurality of search results along with a request to renew thesearch based on the indication of the remainder of the first pluralityof search results.
 15. The computer-implemented method of claim 14,further comprising: determining at least one characteristic of at leastone result of the remainder of the first plurality of search results,wherein transmitting the indication of the remainder of the firstplurality of search results comprises transmitting an indication of theat least one characteristic of the at least one result of the remainderof the first plurality of search results.
 16. The computer-implementedmethod of claim 1, further comprising: causing to be output to the user,along with the first plurality of search results and for each result ofthe first plurality of search results, an interface input mechanism foridentifying a search result as an undesired search result.
 17. Thecomputer-implemented method of claim 1, further comprising: causing tobe output to the user an interface input mechanism for indicating acharacteristic of an undesired search result that renders the undesiredsearch result to be considered undesirable.
 18. The computer-implementedmethod of claim 1, wherein outputting comprises: causing the secondplurality of search results to be output to the user.
 19. Thecomputer-implemented method of claim 1, further comprising: causing tobe output to the user an interface input mechanism for allowing the userto request that the first undesired search result be added back to thesecond plurality of search results.
 20. The computer-implemented methodof claim 19, further comprising: receiving a request of the user, therequest comprising a request to add the first undesired search result tobe added back to the second plurality of search results; adding, uponreceiving the request, the first undesired search result and the secondundesired search result to be added back to the second plurality ofsearch results, thereby determining a third plurality of search results;and causing the third plurality of search results to be output to theuser.
 21. The computer-implemented method of claim 20, wherein the thirdplurality of search results comprises the first plurality of searchresults.
 22. The computer-implemented method of claim 1, wherein thestep of causing the second plurality of search results to be outputoccurs within three seconds of the step of receiving an input of a userassociated with the first plurality of search results, the inputindicating at least one result of the first plurality of search resultsto be removed from the first plurality of search results.
 23. Anon-transitory computer-readable medium storing instructions fordirecting a processor to perform a method, the method comprising:determining, by a processor of a computing device operable to modifysearch results, a first plurality of search results of a searchperformed by a search tool based on a first instance of search termsentered by a user associated with the search; receiving, by theprocessor, an input of the user, the input indicating at least oneresult of the first plurality of search results to be removed from thefirst plurality of search results, thereby receiving an indication of atleast one first currently undesired search result of the first pluralityof search results; removing the currently undesired search result fromthe first plurality of search results; determining, by the processor, aremainder of the first plurality of search results to be the remainingsearch results of the first plurality of search results after the firstcurrently undesired search result has been removed; identifying,dynamically for the search by the processor and based on the inputindicating the at least one result, a characteristic of the at least onefirst undesired search result by identifying a plurality ofcharacteristics corresponding to the first undesired search result andselecting the characteristic from the plurality of characteristics,wherein identifying dynamically comprises identifying based on the firstinstance of search terms entered by the user and without the searchbeing rerun after the input is received; analyzing, dynamically for thesearch and by the processor, the remainder of the first plurality ofsearch results to determine whether any search results of the firstplurality of search results correspond to the characteristic;determining, dynamically and by the processor, at least one searchresult of the remainder of the first plurality of search results thatcorresponds to the characteristic to be an at least one second currentlyundesired search result; removing, dynamically and by the processor, theat least one second currently undesired search result from the firstplurality of search results, thereby determining a second plurality ofsearch results; and causing the second plurality of search results to beoutput.
 24. A computer-implemented method, comprising: determining, by aprocessor of a computing device operable to modify search results, afirst plurality of search results of a search performed by a searchtool; receiving, by the processor, an input of a user associated withthe first plurality of search results, the input indicating a preferencefor a maximum popularity ranking of a search result, in accordance witha ranking scheme in which a higher popularity ranking indicates a morepopular search result than does a lower popularity ranking, wherein theinput defines an undesirable search result as a search result which isassociated with a popularity ranking higher than the maximum popularityranking; analyzing, by the processor, the first plurality of searchresults to identify any search results corresponding to a popularityranking higher than the maximum popularity ranking, thereby analyzingthe first plurality of search results to identify any undesirable searchresults; removing, by the processor, from the first plurality of searchresults, any search results that correspond to a popularity rankinghigher than the maximum popularity ranking, thereby determining a secondplurality of search results which does not include any undesirablesearch results; and causing the second plurality of search results to beoutput.
 25. The computer-implemented method of claim 24, wherein theinput is received prior to an outputting of the first plurality ofsearch results to the user.
 26. The computer-implemented method of claim24, further comprising: outputting the first plurality of search resultsto the user.
 27. The computer-implemented method of claim 26, whereinoutputting the first plurality of search results to the user comprisesoutputting the first plurality of search results to the user beforereceiving the input.
 28. A non-transitory computer-readable mediumstoring instructions for directing a processor to perform a method, themethod comprising: determining a first plurality of search results of asearch performed by a search tool; receiving an input of a userassociated with the first plurality of search results, the inputindicating a preference for a maximum popularity ranking of a searchresult, in accordance with a ranking scheme in which a higher popularityranking indicates a more popular search result than does a lowerpopularity ranking, wherein the input defines an undesirable searchresult as a search result which is associated with a popularity rankinghigher than the maximum popularity ranking; analyzing the firstplurality of search results to identify any search results correspondingto a popularity ranking higher than the maximum popularity ranking,thereby analyzing the first plurality of search results to identify anyundesirable search results; removing from the first plurality of searchresults any search results that correspond to a popularity rankinghigher than the maximum popularity ranking, thereby determining a secondplurality of search results which does not include any undesirablesearch results; and causing the second plurality of search results to beoutput.